Identification and analysis of movement using sensor devices

ABSTRACT

Disclosed are various embodiments for using sensor devices that include accelerometers and gyroscopes to detect and monitor movement of a body. The sensor devices may be coupled to a body in a predefined arrangement, where a sensor device is positioned on a particular portion of the body in accordance with the predefined arrangement. The sensor devices measure a position of the portion of the body to which the band is secured during movement and communicate the position to a client device to be used in generating and updating a three-dimensional reconstruction of the movement performed in near-real-time. Further, the client device may determine whether the movement performed conforms to a predefined movement and, in response to the movement not conforming to the predefined movement, a suggested change in the movement is identified that, if performed, would conform to the predefined movement.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of and priority to U.S. ProvisionalPatent Application No. 62/216,487 entitled “IDENTIFICATION AND ANALYSISOF MOVEMENT USING SENSOR DEVICES,” filed on Sep. 10, 2015, which isincorporated herein by reference in its entirety.

BACKGROUND

Positions and orientations of fixed, rigid bodies, such as a kinematicchain of a robotic manipulator, are tracked using Denavit-Hartenbergconvention and other similar methods. However, detecting position andorientation of various parts of a human remain problematic as size andmakeup of one body may vary from another. Computer vision is commonlyemployed to detect movement of person's body. However, computer visionrequires an imaging device, such as a camera. Additionally, computervision requires a system capable of detecting visual identifiers thatmight not always be accurately detected.

There are various reasons for needing to understand and analyze themovement of a person. In the medical field, screening movement patternsto identify limitations, examine range of motion after an injury, ordocument historical progression is often needed for diagnosis, aid intreatment, and/or rehabilitation. Traditionally, mobility, stability,and coordination of a patient are assessed by doctors, physicaltherapists, and/or other qualified healthcare professionals at a medicalfacility. However, there is a great need to care for individuals inremote areas, where access to specialists can be difficult because ofphysical, economic, and geographic boundaries. Where distance is not abarrier, often age or disability may make it difficult for an individualto travel to a medical facility for diagnosis or treatment. To providethe best outcome for the patient, improved techniques are sought forboth a clinical environment and remote extension.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the present disclosure can be better understood withreference to the following drawings. The components in the drawings arenot necessarily to scale, with emphasis instead being placed uponclearly illustrating the principles of the disclosure. Moreover, in thedrawings, like reference numerals designate corresponding partsthroughout the several views.

FIG. 1 is a drawing of a networked environment that includes sensordevices positioned on a body, a client device, and a computingenvironment according to various embodiments of the present disclosure.

FIG. 2 is a drawing of a body performing a movement where sensor data isused to generate a three-dimensional reconstruction of the movement inthe client device of FIG. 1 according to various embodiments of thepresent disclosure.

FIG. 3 is a drawing of another client device that may be employed toshow a three-dimensional reconstruction of the movement of FIG. 2according to various embodiments of the present disclosure.

FIG. 4 is a drawing of a body performing a movement where sensor data iscollected to generate a three-dimensional reconstruction of the movementin a client device according to various embodiments of the presentdisclosure.

FIG. 5 is a drawing of a client device with two three-dimensionalreconstructions that show the movement of FIG. 4 in view of a predefinedmovement according to various embodiments of the present disclosure.

FIGS. 6-7 are drawings of a client device with two three-dimensionalreconstructions that show a movement performed or being performed inview of another predefined movement according to various embodiments ofthe present disclosure.

FIG. 8 is a flowchart illustrating one example of functionalityimplemented as portions of the sensor devices of FIG. 1 according tovarious embodiments of the present disclosure.

FIG. 9 is an example of pseudocode for reading sensor data according tovarious embodiments of the present disclosure.

FIG. 10 is a flowchart illustrating one example of functionalityimplemented as portions of the client device of FIG. 1 according tovarious embodiments of the present disclosure.

FIG. 11 is an example of pseudocode for processing a sketch according tovarious embodiments of the present disclosure

FIG. 12 is a flowchart illustrating one example of functionalityimplemented as portions of the computing environment of FIG. 1 accordingto various embodiments of the present disclosure.

FIG. 13 is a schematic block diagram that provides one exampleillustration of a sensor device of employed in the networked environmentof FIG. 1 according to various embodiments of the present disclosure.

FIG. 14 is a schematic block diagram that provides one exampleillustration of a client device of employed in the networked environmentof FIG. 1 according to various embodiments of the present disclosure.

FIG. 15 is a schematic block diagram that provides one exampleillustration of a computing environment employed in the networkedenvironment of FIG. 1 according to various embodiments of the presentdisclosure.

DETAILED DESCRIPTION

The present disclosure relates to using sensors to identify movements,such as body movements performed by a human, animal, humanoid, animoid,or other object capable of performing various types of movements. Asnoted above, there are various reasons for needing to understand andanalyze the movement of a person. In the medical field, screeningmovement patterns to identify limitations, examine range of motion afteran injury, or document historical progression is often needed fordiagnosis, aid in treatment, and/or rehabilitation. Traditionally,mobility, stability, and coordination of a patient are assessed bydoctors, physical therapists, and/or other qualified healthcareprofessionals at a medical facility. However, there is a great need tocare for individuals in remote areas, where access to a specialist canbe difficult because of physical, economic, and geographic boundaries.Where distance is not a barrier, often age or disability may make itdifficult for an individual to travel to a medical facility fordiagnosis or treatment. To provide the best outcome for the patient,improved techniques are sought for both a clinical environment andremote extension.

Previous technological applications have analyzed movement using digitalimage processing and computer vision. For example, users may be requiredto position markers or other visual identifiers on their body, and acamera and computer system tracks the movement when the markers or othervisual identifiers are detected and the position has changed from afirst frame to a second frame. However, these systems are limited incapability as they are limited to current technologies in digital imageprocessing, require a camera, and the accuracy in detecting bodymovement is not ideal. For example, it is difficult to identify smallmovements in various joints, such as the knee.

Accordingly, it is beneficial for a system to be capable of monitoringmovement without having to utilize visual analysis. According to variousembodiments of the present disclosure, various sensor devices can bepositioned on a body of a person, animal, humanoid, animoid, or othermoveable body to accurately track movement as will be described. Eachsensor device may be coupled to a band or other attachment that securesthe sensor device to a position on the body. Additionally, each sensordevice may include circuitry that measures a position of the portion ofthe body to which the band is secured during a movement performed by theportion of the body. In various embodiments, each sensor device maycomprise a microcontroller, an accelerometer, a gyroscope, acommunication module, and/or other components as may be appreciated.

In one embodiment, at least one of the sensor devices may be dedicatedas a “master” sensor device, where other ones of the sensor devicescomprise “slave” sensor devices. The slave sensor devices may collectsensor data and communicate the sensor data to the master sensor device.The master sensor device may communicate the sensor data, as well assensor data collected by the master sensor device itself, to a computingdevice for processing. In some embodiments, individual sensor devicescan independently communicate data to the client device and/or thecomputing environment. In various embodiments, the computing devicecomprises a client device, such as a smartphone, a tablet computingdevice, a laptop computing device, a smartwatch, or other computingdevice as will be described.

Further, in various embodiments, the sensor devices may be placed inaccordance with a predefined arrangement stored in memory, such that acomputing device can correlate each of the sensor devices to arespective position of a body. In one embodiment, assuming the body isthat of a person, a first sensor device may be positioned at a firstshoulder of the body; a second sensor device may be positioned at asecond shoulder of the body; a third sensor device may be positioned ata first elbow of the body; a fourth sensor device may be positioned at asecond elbow of the body; a fifth sensor device may be positioned at afirst wrist of the body; a sixth sensor device may be positioned at asecond wrist of the body; a seventh sensor device may be positioned at afirst knee of the body; an eighth sensor device may be positioned at asecond knee of the body; a ninth sensor device may be positioned at afirst ankle of the body; a tenth sensor device may be positioned at asecond ankle of the body; an eleventh sensor device may be positioned ata first side of hips of the body; and a twelfth sensor device may bepositioned at a second side of hips of the body. As may be appreciated,in alternative embodiments, a lesser amount of sensor devices may beutilized to accurately estimate a position of a portion of a body. Forexample, position estimation may be improved using a lesser amount ofsensor devices based on correlations made between different portions ofbody parts (e.g., shoulder position based on position of elbow). To thisend, other arrangements of sensor devices may be utilized, asapplicable. In some embodiments, additional sensors may be located atthe forearm, hand, thigh, foot, neck, and/or head to further markmovement. In other embodiments, sensor devices can be attached to anobject coupled to the body, such as an artificial limb or otherprosthetic; sports equipment, such as a golf club, vaulting pole,baseball bat, etc.; or an object that may decouple from the body, suchas a baseball.

As noted above, the sensor devices may include a communication module,such as a Bluetooth® module, a Bluetooth® low-energy module, a ZigBee®module, a near field communication (NFC) module, or other suitablecommunication module capable of communicating with another computingdevice. In various embodiments, the sensor devices, or a master sensordevice, may be configured to communicate the sensor data to a clientdevice asynchronously. The client device may include a hardwareprocessor, an operating system, and one or more applications(hereinafter client application or client applications) capable ofexecution on the client device by the hardware processor. In variousembodiments, the client application is configured to access the sensordata measured by the sensor devices. The sensor data may include theposition of the portion of the body during the movement performed.

In some embodiments, the client application is configured to generateand update a three-dimensional reconstruction of the movement performedfor viewing in a display, as will be described. Further, the clientapplication may be configured to determine whether the movement of thebody conforms to a predefined movement as well as identify a change inthe movement that, if performed, would conform to the predefinedmovement. In various embodiments, the client application may communicatethe sensor data to a remote computing device, such as a server, over anetwork for remote processing, as will be described.

As will be recognized herein, aspects of the embodiments havefar-reaching applications. For example, sensor devices may be used in aclinical setting to identify movements of an individual before and/orafter a medical procedure or diagnosis. The data may be used to evaluatethe mobility, stability, coordination of a patient, or other diagnosticneeds. Another example includes using the systems described herein intelemedicine applications, for example, where a medical professionalcreates predefined movements, analyzes performed movements, and performsdiagnoses and assignments remotely. Another possible use includes sportsand sports medicine, where an athlete is evaluated to improve techniqueor prevent injury. Further examples include employing the systemsdescribed herein to monitor movements of humanoid devices, animoiddevices, or other robots, for example, to improve gait technique duringrobotic development. This disclosure is not limited to the medical andsports fields or humans and may be extended to include analysis ofanimals or other animate systems.

In the following discussion, a general description of the system and itscomponents is provided, followed by a discussion of the operation of thesame.

With reference to FIG. 1, shown is a drawing of a body 100 of a personwith a plurality of sensor devices 103 a . . . 1031 (collectively sensordevices 103) positioned thereon. In various embodiments, the sensordevices 103 may be secured to the body 100 using bands 106 a . . . 106 k(collectively bands 106), such as an elastic band that conforms tomovement of the body 100 without detachment. Individual sensor devices103 may be secured to the band 106, for example, by applying an adhesiveto a sensor device 103, affixing the sensor device 103 to the band 106using Velcro®, positioning the sensor device 103 in a receptacle of theband 106 (e.g., a pouch or a container), sewing the sensor device 103 tothe band 106, or applying another suitable affixing technique. In someembodiments, the sensor device may be secured to clothing or adhereddirectly to the body.

For reference, the components of sensor device 103 a are shown inFIG. 1. As noted above, the sensor device 103 a may comprise circuitry,such as processing circuitry. In various embodiments, the sensor device103 a may comprise a microcontroller 109, an accelerometer 112, agyroscope 115, a magnetometer 117, a communication module 118, and/orother components. In further embodiments, the sensor device 103 a maycomprise other circuitry, such as an altimeter.

The microcontroller 109 may include circuitry, for example, on a singleintegrated circuit, such as a processor core, memory, and programmableinput and output peripherals. To this end, the microcontroller 109 maybe preconfigured to control the operations of the components of thesensor device 103 a, such as the accelerometer 112, the gyroscope 115,and/or the communication module 118. In some embodiments, themicrocontroller 109 performs control operations at the behest of aclient device 121 or a computing environment 124, as will be discussed.Further, the microcontroller 109 may be preconfigured to obtain andstore sensor data 127 a . . . 127 c (collectively sensor data 127) inlocal memory. The sensor data 127 a stored locally on the sensor device103 a may include, for example, position data collected from theaccelerometer 112, the gyroscope 115, as well as data received by thecommunication module 118.

The microcontroller 109 may be further configured to determine anorientation and position of the sensor device 103 a in athree-dimensional space, or raw sensor data 127 a may be communicated tothe client device 121 to determine the orientation or position. Thegyroscope 115 is configured to measure the rate of change of an angularposition of the sensor device 103 a over time, also referred to asangular velocity, in a unit of degrees per second. In other words, agyroscope 115 provides the derivative of the angular position over time:

$\begin{matrix}{\overset{.}{\theta} = \frac{d\;\theta}{d\; t}} & \left( {{eq}.\mspace{14mu} 1} \right)\end{matrix}$

where {dot over (θ)} is the metric obtained from the gyroscope 115,i.e., the angular position over time. As the metric obtained from thegyroscope 115 is the derivative of the angular position, the angularposition can be calculated by:θ(t)=∫₀ ^(t){dot over (θ)}(t)dt≈Σ ₀ ^(t){dot over (θ)}(t)T _(s)  (eq. 2)where θ(t) is the angular position at a given point in time, t, andT_(s) is a sampling frequency. In various embodiments, the samplingfrequency may be between 100 Hz and 200 Hz, although other samplingfrequencies may be employed. Sampling frequencies employed between 100Hz and 200 Hz may be ideal to eliminate errors caused by minor bumps andvibrations.

As the gyroscope 115 in some implementations may have difficultyreturning to zero when the sensor device 103 a returns to its originalposition, data from the accelerometer 112 may be utilized. In someembodiments, a filter may be applied to determine an angle for each axisthat may be used to ultimately calculate an orientation and position ofthe sensor device 103 a in a three-dimensional space. In someembodiments, a Kalman filter may be employed. In other embodiments, acomplimentary filter may be employed where an angle for a given axis maybe calculated via:α=0.98*(α+Data_(Gyroscope) *dt)+0.02*(Data_(Accelerometer))  (eq. 3)where α is the angle, 0.98 is the gravitational constant,Data_(Gyroscope) is data from a gyroscope 115, and Data_(Accelerometer)is data from an accelerometer 112.

In various embodiments, the components of the sensor device 103 a arecapable of measuring a movement having six-degrees-of-freedom (6DoF) ina three-dimensional space. In other words, each sensor device 103 maymonitor changes in its position, such as changes in left movement, rightmovement, upwards movements, downwards movement, forward movement,backwards movement, pitch, roll, and yaw. To this end, in someembodiments, the sensor device 103 a may comprise three accelerometers112 and three gyroscopes 115.

The communication module 118 includes a network interface componentcapable of sending and receiving data to other devices. To this end, thecommunication module 118 may include a Bluetooth® module, a Bluetooth®low-energy module, a ZigBee® module, a wireless fidelity (Wi-Fi) module,a near field communication (NFC) module, or other suitable module. Usingthe communication module 118, the microcontroller 109 may causetransmission of the sensor data 127 a from the sensor device 103 a to acomputing device, such as the client device 121 (e.g., as sensor data127 b).

A networked environment 130 is also described in accordance with variousembodiments. The networked environment 130 may be described as includingthe sensor devices 103, the computing environment 124, and the clientdevice 121. The computing environment 124 and the client device 121 arein data communication with each other via a network 133. The network 133includes, for example, the Internet, intranets, extranets, wide areanetworks (WANs), local area networks (LANs), wired networks, wirelessnetworks, or other suitable networks, etc., or any combination of two ormore such networks. For example, such networks may comprise satellitenetworks, cable networks, Ethernet networks, and other types ofnetworks.

The computing environment 124 may comprise, for example, a servercomputer or any other system providing computing capability.Alternatively, the computing environment 124 may employ a plurality ofcomputing devices that may be arranged, for example, in one or moreserver banks or computer banks or other arrangements. Such computingdevices may be located in a single installation or may be distributedamong many different geographical locations. For example, the computingenvironment 124 may include a plurality of computing devices thattogether may comprise a hosted computing resource, a grid computingresource and/or any other distributed computing arrangement. In somecases, the computing environment 124 may correspond to an elasticcomputing resource where the allotted capacity of processing, network,storage, or other computing-related resources may vary over time.

Various applications and/or other functionality may be executed in thecomputing environment 124 according to various embodiments. Also,various data is stored in a data store 136 that is accessible to thecomputing environment 124. The data store 136 may be representative of aplurality of data stores 136 as can be appreciated. The data stored inthe data store 136, for example, is associated with the operation of thevarious applications and/or functional entities described below.

The components executed on the computing environment 124, for example,include a data processing application 139, and other applications,services, processes, systems, engines, or functionality not discussed indetail herein. Generally, the data processing application 139 isexecuted to provide remote data services for sensor devices 103 andclient devices 121. To this end, in some embodiments, the dataprocessing application 139 is configured to receive and store sensordata 127 c from the sensor devices 103 via the client device 121 andperform remote processing on the sensor data 127. In some embodiments,the data processing application 139 may generate or updatethree-dimensional reconstructions of the body 100 for display in one ormore client devices 121. Further, the data processing application 139may be configured to permit a user to establish or update ideal movementdata 142 that may be used to determine, for example, whether a movementof the body 100 conforms to a predefined movement.

In one example, a medical professional may use his or her own batch ofsensor devices 103 to remotely define movements assigned to a patient.For example, the medical professional may attach sensor devices 103 tohis or her own body and perform a series of movements. The dataprocessing application 139 can monitor the movements performed by themedical professional and store the movements as ideal movement data 142.A patient may access the client application 158 to view the movementsprescribed to him or her by the medical professional, e.g., by viewing athree-dimensional reconstruction of the movement. The patient canattempt to perform the movement and the data processing application 139can determine whether the movement conforms to the ideal movement data142 created by the medical professional. Accordingly, the medicalprofessional can determine (a) whether the patient has performed his orher assigned movements; (b) whether the movements performed by the userconform to the movement assigned by the medical professional; and (c) ifthe movement performed by the patient does not conform to the movementassigned by the medical professional, determine why the movementperformed by the patient does not conform to the movement assigned bythe medical professional.

In addition, the data processing application 139 can identify whymovements performed by a user do not conform to ideal movement data 142and generate suggested changes 145 to the movements that, if performed,might cause the movements to conform to ideal movement data 142.

The data stored in the data store 136 includes, for example, idealmovement data 142, suggested changes 145, sensor arrangements 148,three-dimensional reconstructions 150 a . . . 150 b (collectivelythree-dimensional reconstructions 150), user data 152, and potentiallyother data. The ideal movement data 142 may include sensor datacollected during an ideal movement performed by a body 100. For example,a professional basketball player may wear sensor devices 103 whileperforming a jump shot, a professional baseball player may wear sensordevices 103 while pitching, or a professional pole vaulter may wearsensor devices 103 while performing an ideal jump. Sensor data 127 maybe collected during these ideal movements and stored in the data store136 as ideal movement data 142.

In other embodiments, a user may define ideal movement data 142 to beperformed by another user. For example, a medical professional maydefine ideal movement data 142 for a patient recently having a ulnarcollateral ligament reconstruction (UCL) treatment. The ideal movementdata 142 may include data collected from sensor devices 103 when themedical professional, or other body 100, performs movements, such asrotating the shoulder up, down, left, and right, simulating a pitchingmotion, etc. The patient may be assigned the movements to be performedand the actual movements performed by the patient may be compared to theideal movement data 142.

The suggested changes 145 includes data indicative of why movementsperformed by a user do not conform to ideal movement data 142 as well assuggested movements that, if performed, might cause the movements toconform to ideal movement data 142. Moving along, sensor arrangements148 may include one or more predefined arrangements of sensor devices103 capable of correlating a particular sensor to a particular positionon a body 100. An example database table of a predefined arrangement ofsensor devices 103 that may be stored in the data store 136 is shownbelow in Table 1:

TABLE 1 Example Database Table of a Predefined Arrangement of SensorDevices 103 int Sensor_Identifier var Position int User_ID 012333457Left Wrist 042857 012333458 Right Wrist 042857 012333459 Left Elbow042857 . . . . . . . . .

As shown above in Table 1, a sensor device 103 may be associated with asensor identifier, which may include a unique identifier. The databasetable may be queried using the sensor identifier to identify theposition of the corresponding sensor device 103 on the body 100.Further, the sensor identifier may be stored in association with a useridentifier, which may include an identifier that uniquely identifies auser of the client device 121, e.g., the wearer of the sensor devices103. Accordingly, different arrangements of sensor devices 103 may beassigned to different bodies 100, whether persons, animals, humanoids,animoids, etc. In various embodiments, the sensor arrangements 148 maybe predefined by a user of the client device 121 (e.g., a wearer of thesensor devices 103) or by another user (e.g., a medical professional, acoach, or other person).

The three-dimensional reconstructions 150 include graphicalrepresentations of an electronically generated body performing amovement that may be generated by the client device 121 or the computingenvironment 124. In various embodiments, the three-dimensionalreconstructions 150 may be generated by a three-dimensional model andfigure generator in the form of an application, service, or library. Inembodiments where an application generates the three-dimensionalreconstruction 150 comprises Poser®, MakeHuman™, DAZ 3D, Misfit Model3D, Mixamo® Fuse, XnaLara Realtime Posing Program, GLLara, or Vizago.

User data 152 may include data corresponding to the wearer of the sensordevices 103 which may be the user of the client device 121. To this end,the sensor data 127 c collected during a movement performed by a usermay be stored in association with the user as well as a time themovement occurred.

The client device 121 is representative of a plurality of client devices121 that may be coupled to the network 133. The client device 121 maycomprise, for example, a processor-based system such as a computersystem. Such a computer system may be embodied in the form of a desktopcomputer, a laptop computer, personal digital assistants, cellulartelephones, smartphones, set-top boxes, music players, web pads, tabletcomputer systems, game consoles, electronic book readers, or otherdevices with like capability. The client device 121 may include adisplay 155. The display 155 may comprise, for example, one or moredevices such as liquid crystal display (LCD) displays, gas plasma-basedflat panel displays, organic light emitting diode (OLED) displays,electrophoretic ink (E ink) displays, LCD projectors, or other types ofdisplay devices, etc.

The client device 121 may be configured to execute various applicationssuch as a client application 158 and/or other applications. The clientapplication 158 may be executed in a client device 121, for example, toaccess content served up by the computing environment 124, otherservers, and/or the sensor devices 103 to render a user interface 169 onthe display 155. To this end, the client application 158 may comprise,for example, a dedicated application, such as a mobile or clientapplication, a web browser, etc., and the user interface 169 maycomprise an application screen, a network page, etc. The client device121 may be configured to execute applications beyond the clientapplication 158 such as, for example, email applications, socialnetworking applications, word processors, spreadsheets, and/or otherapplications.

Referring next to FIG. 2, shown is a drawing of a body 100 performing amovement where sensor data 127 is collected during the movement andcommunicated to a client device 121. The client application 158 may beexecuted in the client device 121 to access and store the sensor data127. In various embodiments, the client application 158 may beconfigured to generate a three-dimensional reconstruction 150 a of themovement in near-real-time as the movement is performed. To this end, athree-dimensional reconstruction 150 a may be rendered in the display155 as the movement is performed, or thereafter.

As may be appreciated, the three-dimensional reconstruction 150 a maynot include a video feed of the body 100 performing the movement.Instead, the three-dimensional reconstruction 150 a includes graphicalrepresentations of an electronically generated body performing amovement. In one embodiment, a single three-dimensional reconstruction150 a may be shown in the display 155 that mirrors the movementsperformed by the body 100. In another embodiment, a firstthree-dimensional reconstruction 150 a is shown along a secondthree-dimensional reconstruction 150 b. For example, the firstthree-dimensional reconstruction 150 a may reflect a movement performedby the body 100 while the second three-dimensional reconstruction 150 breflects a reconstruction of ideal movement data 142.

To this end, the movement performed by the user may be compared to amovement performed by himself or herself in the past or may be comparedto a movement performed by another person, such as a movement performedby a medical professional when assigning a movement or a movementperformed by an athlete. The first three-dimensional reconstruction 150a may be overlaid on the second three-dimensional reconstruction 150 b,or vice versa. In other embodiments, the first three-dimensionalreconstruction 150 a and the second three-dimensional reconstruction 150b may be shown side by side or one or top of the other in the display155.

Moving on to FIG. 3, shown is another example of a client device 121that may be used to display a three-dimensional reconstruction while amovement is performed or thereafter. In one embodiment, the clientdevice 121 may comprise a smartwatch that may show one or morethree-dimensional reconstructions 150 in the display 155 that mirrorsthe movements performed by the body 100.

Turning now to FIG. 4, shown is another drawing of a body 100 performinga movement where sensor data 127 is collected during the movement andcommunicated to a client device 121. In the non-limiting example of FIG.4, a golf swing is shown as a movement that may be performed to collectsensor data 127 during the movement. The sampling period to collectsensor data 127 may include a start of the golf swing and an end of thegolf swing, as may be appreciated. The sampling rate may include apredefined time interval at which a position and/or orientation isdetermined and/or obtained from each of the sensor devices 103. Forexample, a position and/or orientation of the sensor devices 103 may bedetermined and/or obtained every 0.5 ms, or other suitable timeinterval.

As shown in FIG. 5, the sensor data 127 collected during the golf swingmovement may be used to render a three-dimensional reconstruction 150 ain near-real-time for encoding in a user interface 169 rendered on adisplay 155, either as the movement is performed or thereafter. To thisend, an athlete is able to view his or her movement to improvetechnique. In the non-limiting example of FIG. 5, the client application158 causes a rendering of a first three-dimensional reconstruction 150 aside-by-side with a second three-dimensional reconstruction 150 b. Thefirst three-dimensional reconstruction 150 a may be generated using thesensor data 127 collected during the movement while the secondthree-dimensional reconstruction 150 b may be generated using idealmovement data 142.

The client application 158 may be configured to analyze differencesbetween the sensor data 127 collected during the movement and the idealmovement data 142. As noted above, the ideal movement data 142 mayreflect sensor data 127 collected by a person who performed an idealmovement. Accordingly, a comparison of sensor data 127 to sensor data127 is performed. In other words, sensor data 127 collected when a firstperson performed a movement is compared to sensor data 127 collectedwhen a second person performed the same movement. A position of a sensordevice 103 on the first person may be compared to a position of a sensordevice 103 on a second person at a same point in time during themovement. For example, given the time t, a position of sensor device 103for a first person may be compared to a position of a sensor device 103for a second person and, if the differences in position exceed apredefined threshold, a user of the client device 121 may be notified ofthe difference.

As a non-limiting example, a wrist position of a professional golfer maybe monitored during a golf swing by placing a sensor device 103 at thewrist of the professional golfer. Sensor data 127 is collected from thesensor device 103 during the golf swing to measure a position andorientation of the sensor device 103 (i.e., the wrist) at a predefinedsampling rate. This data may be stored as ideal movement data 142.Similarly, a wrist position of an amateur golfer may be monitored duringa golf swing by placing a sensor device 103 at the wrist of the amateurgolfer. Sensor data 127 is also collected from the sensor device 103during the golf swing to measure a position and orientation of thesensor device 103 at the same predefined sampling rate. The sensor data127 collected from the amateur golfer may be compared to that of theprofessional golfer to identify differences between the positions of thewrist during the golf swing. If a difference in the position exceeds apredefined threshold (e.g., that allows for some degree of error), theuser of the client device 121 may be notified. Further, a suggestedchange 145 may be shown to the user of the client device 121.

A function may be used to obtain a position (and/or orientation) of asensor device 103 at a given time:

float sensor_position(int time, int device_id) {  // EstablishConnection with Database   $conn = new mysqli([AUTHENTICATION_INFO]); // Query Database to Find Position  $query_string = “SELECT * FROMtable WHERE     device_id=’”+$device_id+”’ AND time=’”+$time+”’”; $position = $conn->query($query_string);  // Return Position return(position); }

To this end, a first position of a first sensor device 103 (e.g., forthe amateur golfer) may be obtained using the function and a secondposition of a second sensor device 103 (e.g., for the professionalgolfer). The positions may then be compared. If the first sensor device103 is within a predefined threshold, the position of the amateur golferis nearly identical to that of the professional golfer. However, if thefirst sensor device 103 is not within a predefined threshold, theposition of the amateur golfer is not identical to that of theprofessional golfer. Accordingly, the user of the client device 121 maybe notified to adjust his or her wrist position at a given point in timeduring the swing using a suggested change 145.

In various embodiments, a time at which a position of a first group ofsensor devices 103 (e.g., for a first person) is compared to a secondgroup of sensor devices 103 (e.g., for a second person) may be specifiedby a slider bar 503. As a result, a user of the client device 121 mayaccess and view suggested changes 145 during a frame-by-frame analysisof the movement. Alternatively, the user may view the movement in itsentirety at a real-time play speed to see suggested changes 145 atdifferent times during playback of the movement. In some embodiments,the playback of the movement may be sped up or slowed down.

Further, the client application 158 may be executed in the client device121 and may store the sensor data 127 for later access and/orcommunicate the sensor data 127 over the network 133 to be storedremotely by the computing environment 124. The client application 158may be configured to generate the three-dimensional reconstructions 150or the computing environment 124 may remotely generate thethree-dimensional reconstructions 150 and provide them to the clientdevice 121 for rendering in the display 155.

As may be appreciated, a three-dimensional reconstruction 150 may notinclude a video feed of the body 100 performing the movement. Instead,the three-dimensional reconstructions 150 include graphicalrepresentations of an electronically generated body performing amovement. In one embodiment, a single three-dimensional reconstruction150 may be shown in the display 155 that mirrors the movements performedby the body 100. In another embodiment, a first three-dimensionalreconstruction 150 a is shown along with a second three-dimensionalreconstruction 150 b. For example, the first three-dimensionalreconstruction 150 a may reflect a movement performed by the body 100while the second three-dimensional reconstruction 150 b reflects areconstruction of ideal movement data 142. To this end, the movementperformed by the user may be compared to a movement performed by himselfor herself in the past or may be compared to a movement performed byanother person, such as a movement (assigned performed by a medicalprofessional when assigning a movement or a movement performed by anathlete.

Moving on to FIG. 6, shown is a drawing of a client device 121 used torender three-dimensional reconstructions 150 a . . . 150 b. In variouscircumstances, a person may have a surgery and, as a result of thesurgery, a movement performed by the person is changed. For example, aperson might walk, run, or perform other exercises differently after aknee surgery.

Accordingly, in various embodiments, the client application 158 may beexecuted to compare sensor data 127 collected for a patient during amovement (e.g., gait, running, or other movement) to sensor data 127collected by a representative patient to show expected differencesbetween the movements. For example, a notification may be presented inthe client application 158 that indicates that knee flexibility will beimproved.

Similarly, in FIG. 7, the client application 158 may be executed tocompare sensor data 127 collected by a medical professional to remotelyassign a movement to a patient. The sensor data 127 may be collected bythe medical professional wearing sensor devices 103 or the sensor data127 may be accessed from a database of previously analyzed movements.For example, a medical professional may use sensor data 127 collectedfrom another patient to assign a movement to a patient.

Either the patient and/or the medical professional may use a clientdevice 121 to access three-dimensional reconstructions 150 generatedusing the sensor data 127. As a result, the exercise performed by apatient may be compared to the assigned exercise to see if the patientcan conform his or her movement to that of the assigned exercise. Theclient application 158 may detect differences between the two sets ofsensor data 127 and communicate the differences to the patient and/orthe medical profession, for example, by rendering a suitablenotification in the user interface 169. Additionally, notifications fromthe doctor to patient, or vice versa, may be rendered in the display155, for example, to direct the patient how to improve or to notify thepatient that he or she is performing well.

Referring next to FIG. 8, shown is a flowchart that provides one exampleof the operation of a portion of the microcontroller 109 of the sensordevice 103 according to various embodiments. It is understood that theflowchart of FIG. 8 provides merely an example of the many differenttypes of functional arrangements that may be employed to implement theoperation of the portion of the microcontroller 109 as described herein.As an alternative, the flowchart of FIG. 8 may be viewed as depicting anexample of elements of a method implemented by a microprocessorapplication executable by a hardware processor according to one or moreembodiments.

Beginning with 803, the microcontroller 109 initializes its serial portsor other communication interface, if necessary. For example, somemicrocontrollers 109 communicate with components of the microcontroller109, or other circuitry, using serial port processing. In oneembodiment, the microcontroller 109 communicates with the one or moreaccelerometers 112, gyroscopes 115, magnetometers 117, etc., usingserial ports. To this end, in some embodiments, a microprocessorapplication may perform a function call to a library to initialize theserial ports for communication.

Next, in 806, sensor data 127 is collected from the one or moregyroscopes 115, accelerometers 112, magnetometers 117, etc. Inembodiments where the sensor device 103 is capable of 6DoF, sensor data127 is collected from three gyroscopes 115 and three accelerometers 112.In other embodiments, the sensor device may be capable of 9DoF or more.

Next, in 809, the microcontroller 109 may process the sensor data 127,for example, to determine an orientation and position of the sensordevice 103 in a three-dimensional space. In alternative embodiments, rawsensor data 127 a may be communicated to the client device 121 todetermine the orientation or position remotely. If the microcontroller109 locally processes the sensor data, in some embodiments, themicrocontroller 109 ultimately may determine four quaternions fromsensor data 127 collected from an inertial measurement unit (IMU) thatmay comprise the gyroscopes 115, the accelerometers 112, and/or otherdevices. The output data from the sensors could be quaternion vectors ora three-dimensional position (x, y, z) with an angle of rotation toestablish a rotational matrix. Each quaternion may include afour-element vector that can be used to encode a rotation in athree-dimensional coordinate system. As may be appreciated, thequaternions may be used to calculate orientation and position (orlocation) in a three-dimensional coordinate system and utilized assuitable input to graphics code for three-dimensional reconstruction.

In 812, a determination may be made whether the sensor device 103 onwhich the microcontroller 109 is embedded is a master sensor device. Asnoted above, in some embodiments, at least one of the sensor devices 103in a group of sensor devices 103 may be dedicated as a “master” sensordevice 103, where other ones of the sensor devices 103 are “slave”sensor devices 103. Accordingly, if the sensor device is not designatedas a master sensor device 103, the process proceeds to 815 where thesensor data 127 (as processed) is communicated to a master sensor device103. Thereafter, the process proceeds to completion.

Referring back to 812, if the sensor device 103 on which themicrocontroller 109 is embedded is designated as the master sensordevice the process proceeds to 818 where sensor data 127 is collectedfrom the slave sensor devices 103. Next, in 821, the sensor data 127collected by the components of the master and slave sensor devices 103is communicated to a client device 121, such as a smartphone, a tabletcomputing device, a laptop computing device, a smartwatch, or othercomputing device.

As noted above, the sensor devices 103 include a communication module118, such as a network interface component capable of sending andreceiving data to other devices. To this end, the communication module118 may include a Bluetooth® module, a Bluetooth® low-energy module, aZigBee® module, a Wi-Fi module, a near field communication (NFC) module,or other suitable module. Using the communication module 118, themicrocontroller 109 may cause transmission of the sensor data 127 fromthe sensor device 103 to a computing device, such as the client device121. In embodiments where the communication module 118 includes a Wi-Fimodule, the sensor data 127 may be communicated directly to thecomputing environment 124 using a wireless network.

Turning now to FIG. 9, shown is an example of pseudocode that may beused to configure a microprocessor application executable by a hardwareprocessor of a microcontroller 109 onboard a sensor device 103. Thepseudocode reflects an example of an algorithm that causes performanceof some functions of the sensor device 103 (or the microcontroller 109of the sensor device 103) as described throughout the presentdisclosure. The pseudocode may not reflect a particular softwarelanguage, but may describe functions, function calls, variabledefinitions, or other actions that, if implemented in one or moresoftware languages, configure the microcontroller 109 to perform thefunctions described herein.

Notably, the pseudocode describes including applicable libraries andcreating a variable for device identifier data and quaternion data,which may be embodied as an array of floats. A library included in theapplication may comprise classes, functions, or other calls that permithardware control. For example, an object for “SensorDevice” may becreated where .Initialize( ) is a function that initializes the sensordevice.

In a setup routine, serial ports or other communication interfaces maybe initialized, if necessary. For example, a microcontroller 109 may bemanufactured to communicate with other devices or circuitry using serialport processing. The microprocessor application may perform a functioncall to a suitable library to initialize the serial ports forcommunication. A wait is performed to allow the serial ports toinitialize.

Thereafter, a loop routine is performed to collect data from componentsof the sensor device 103, such as the IMU. For example, a sensoridentifier may be accessed from the IMU and stored at location [0] inarray q, a first quaternion may be accessed from the IMU and stored atlocation [1] in array q, and so forth. The array q is then communicatedover the serial port. In one example, the serial port allows the data tobe communicated to the communication module 118, which then communicatesthe data to the client device 121. The loop continues to repeat untilthe application is exited or terminated based on a predefined condition(e.g., powering off the device, disabling data collection, or othercondition).

Moving on to FIG. 10, shown is a flowchart that provides one example ofthe operation of a portion of the client device 121 according to variousembodiments. It is understood that the flowchart of FIG. 10 providesmerely an example of the many different types of functional arrangementsthat may be employed to implement the operation of the portion of theclient device 121 as described herein. As an alternative, the flowchartof FIG. 10 may be viewed as depicting an example of elements of a methodimplemented by the client application 158 executable by a hardwareprocessor according to one or more embodiments.

As noted above, one or more sensor devices 103 can be positioned on abody 100 of a person, animal, humanoid, animoid, or other moveable bodyto accurately track movement where more sensor devices 103 can producemore granularities in movement detection. Each sensor device 103 may becoupled to a band 106 or other attachment that secures the sensor device103 to a position on the body 100. Additionally, each sensor device 103may include circuitry that measures a position of the portion of thebody 100 to which the band is secured during a movement performed by theportion of the body 100. As shown in FIG. 1, each sensor device 103 maycomprise a microcontroller 109, an accelerometer 112, a gyroscope 115, amagnetometer 117, a communication module 118, and/or other components asmay be appreciated. As at least one of the sensor devices 103 isconfigured to communicate with a client device 121, the clientapplication 158 can process sensor data 127 and generate user interfaces169 that include three-dimensional reconstructions 150 of performedmovements.

The sensor devices 103 may be placed in accordance with a predefinedarrangement stored in memory (e.g., the data store 136), such that acomputing device can correlate each of the sensor devices 103 to arespective position of a body 100. In one embodiment, assuming the body100 is that of a person or a humanoid, a first sensor device 103 a maybe positioned at a first shoulder of the body 100; a second sensordevice 103 b may be positioned at a second shoulder of the body 100; athird sensor device 103 c may be positioned at a first elbow of the body100; a fourth sensor device 103 d may be positioned at a second elbow ofthe body 100; a fifth sensor device 103 e may be positioned at a firstwrist of the body 100; a sixth sensor device 103 f may be positioned ata second wrist of the body 100; a seventh sensor device 103 g may bepositioned at a first knee of the body 100; an eighth sensor device 103h may be positioned at a second knee of the body 100; a ninth sensordevice 103 i may be positioned at a first ankle of the body 100; a tenthsensor device 103 j may be positioned at a second ankle of the body 100;an eleventh sensor device 103 k may be positioned at a first side ofhips of the body 100; and a twelfth sensor device 103 l may bepositioned at a second side of hips of the body 100.

Beginning with 1003, the client device 121 enables the sensor devices103 to collect sensor data 127. In one embodiment, the client device 121sends a suitable control signal to each sensor device 103 that causesthe sensor device 103 to start collecting sensor data 127. In anotherembodiment, the client device 121 sends a suitable control signal to amaster sensor device 103, which then sends suitable control signals toslave sensor devices 103 that causes the sensor devices 103 to startcollecting sensor data 127.

Next, in 1006, the client device 121 initializes the display of one ormore three-dimensional reconstructions 150. To this end, a clientapplication 158 or service may be executed that utilizes graphiclibraries to eventually render a three-dimensional reconstruction 150 ofa body 100, where the three-dimensional reconstruction 150 is updated innear-real-time to mirror movements performed by the body 100 wearing thesensor devices 103 using the three-dimensional reconstruction 150.

In 1009, sensor data 127 is received from the sensor devices 103, forexample, using Bluetooth®, ZigBee®, Wi-Fi, or other suitablecommunication medium. As noted above, the sensor devices 103 may includea communication module 118, such as a Bluetooth® module, a Bluetooth®low-energy module, a ZigBee® module, an NFC module, a Wi-Fi module, orother suitable communication module 118 capable of communicating withanother computing device. In various embodiments, the sensor devices103, or a master sensor device 103, may be configured to communicate thesensor data 127 to a client device 121 asynchronously. The client device121 may include a hardware processor, an operating system, and one ormore applications (hereinafter client application or clientapplications) capable of execution on the client device by the hardwareprocessor. In various embodiments, the client application 158 isconfigured to access the sensor data 127 measured by the sensor devices.The sensor data 127 may include the position of the portion of the body100 during the movement performed, or other data collection during themovement.

In 1012, the sensor data 127 collected from the sensor devices 103 maybe stored in a buffer or in local memory. To this end, in someembodiments, the sensor data 127 may be stored in a database (e.g.,structured query language (SQL) database or SQL-Lite database) of thedata store 136 in association with a time the movement was performed.

Next, in 1015, the sensor data 127 may be processed to determine aposition and/or orientation of each of the sensors devices 103 based onthe sensor data 127 received from each of the sensor devices 103. Forexample, sensor data 127 received from a sensor device 103 at a shouldermay be processed to identify a position or orientation of the sensordevice 103 at the shoulder, and so forth. In some embodiments, thesensor data 127 is processed to monitor changes in a position of acorresponding sensor device 103, such as changes in left movement, rightmovement, upwards movements, downwards movement, forward movement,backwards movement, pitch, roll, and yaw.

Using the processed sensor data 127, in 1018, the three-dimensionalreconstruction 150 is generated, updated, and/or rendered in a display155 of the client device 121. In one embodiment, to update thethree-dimensional reconstruction 150, a programmatic function call orapplication programming interface (API) call is performed to provide agraphics library with an updated position and/or orientation of thecorresponding portion of the body 100.

Further, in some embodiments, the client device 121 (e.g., via theclient application 158) may be configured to determine whether amovement performed by a body 100, having the sensor devices 103 coupledthereon, conforms to an ideal movement, such as a movement assigned by amedical professional, a movement performed by a coach or athlete, orother predefined movement. Accordingly, in 1021, the movement performedby the body 100 having the sensor devices 103 coupled thereon iscompared to ideal movement data 142 representative of an ideal movement.

The ideal movement data 142 may reflect sensor data 127 collected by aperson who performed an ideal movement. As a result, a comparison ofsensor data 127 to sensor data 127 is performed. In other words, sensordata 127 collected when a first person performed a movement is comparedto sensor data 127 collected when a second person performed the samemovement. A position of a sensor device 103 on the first person may becompared to a position of a sensor device 103 on a second person duringat a same point in time during the movement. For example, given the timet, a position of sensor device 103 for a first person may be compared toa position of a sensor device 103 for a second person and, if thedifferences in position exceed a predefined threshold, a user of theclient device 121 may be notified of the difference. As a non-limitingexample, given t, a query is performed to obtain data from the idealmovement data 142.

-   -   SELECT * FROM ideal_movement_data_table WHERE        sensor_id=′[SENSORID]′ AND time=′[TIME t]″

Similarly, given the same time t, a query is performed to obtain datafrom the sensor devices 103 performed by the body 100.

-   -   SELECT * FROM sensor_data_table WHERE sensor_id=′[SENSORID]′ AND        time=′[TIME t]″

As a non-limiting example, Table 2 describes the data from the query tothe “ideal_movement_data_table” and the query to the“sensor_data_table:”

TABLE 2 Sample Results from Two Queries to Compare Sensor PositionsSensor_ID Sensor_Pos User_ID Time Q1 . . . Q4 012333457 Left Wrist042857 t [Q_VALUES] 489611342 Left Wrist 054621 t [Q_VALUES]The row for “sensor_id”=01233457 contains information associated with anideal movement data 142 and the row for “sensor_id”=489611342 containsinformation associated with sensor data 127 obtained from a movementperformed, for example, by a current wearer of the sensor device 103.The values of the two rows stored as columns under “pos” may be comparedto determine whether a first movement conforms to a second movement, orvice versa. The comparison may include accounting for error using apredefined threshold, as may be appreciated.

In 1024, a determination is made whether a first movement (e.g., amovement performed by a wearer of the sensor devices 103) conforms to asecond movement (e.g., ideal movement data 142). If a first movement iswithin a predefined threshold of a second movement, no action may betaken and the process may proceed to 1027. However, in some embodiments,a notification may be generated for rendering on a display 155 thatindicates that the movement performed by a user conforms with idealmovement data 142 (e.g., “Good job, your movement is correct!”).

Referring back to 1024, if the first movement does not conform to thesecond movement (e.g., the first position is not within a predefinedthreshold of the second movement), the process proceeds to 1030 where asuggested change 145 in the movement may be generated for rendering inthe display 155. The suggested change 145 may include, for example, achange in the position of the portion of the body 100 where a particularsensor device 103 is attached that, if performed as suggested, wouldconform to the ideal movement data 142. As may be appreciated, the userof the client device 121 may then attempt to perform a subsequentmovement based on the suggested change 145 to conform to an idealmovement.

The process then proceeds to 1027 where a determination is made whethera terminate condition has been identified. In some embodiments, aterminate condition may occur after a predefined amount of time, after acomplete movement has been performed (e.g., a complete pitch motion, acomplete jump shot, or a complete golf swing), when a user closes theclient application 158, or when a user pauses the collection of sensordata 127 in the client application 158. If a terminate condition is notidentified, the process reverts back to 1009 to continue accessingcollected sensor data 127, for example, to continue updating thethree-dimensional reconstruction 150 and to continue generatingsuggested changes 145 in movement. If a terminate condition isidentified, however, the process may proceed to completion.

Referring next to FIG. 11, shown is an example of pseudocode that may beused to configure the client application 158 ultimately executable by ahardware processor of the client device 121. The pseudocode reflects anexample of an algorithm that causes performance of some functions of theclient device 121 as described throughout the present disclosure. Thepseudocode may not reflect a particular software language, but maydescribe functions, function calls, variable definitions, or otheractions that, if implemented in one or more software languages,configure the client device 121 to perform the functions describedherein.

Notably, the pseudocode describes including applicable libraries toallow access to particular libraries that may be needed to interfacewith particular peripherals of the client device 121, such as thedisplay 155, the serial ports or other communication modules, memory,etc. Various data objects (e.g., JavaScript Object Notation (JSON) dataobjects), screen dimensions, serial ports, and variables may beinitialized. Further, an object model may be initialized to create athree-dimensional reconstruction 150 using a suitable graphics library.In some embodiments, node.js library can be used to generate or modify athree-dimensional reconstruction.

A setup routine may be performed that establishes a size of a graphicaluser interface (e.g., user interface 169), loads images to be used inobject models (e.g., three-dimensional reconstructions), scales theimages to be used in the object models, defines a user interface area(e.g., a bounding box), instantiates serial ports, and stores dataobtained over the serial ports (e.g., sensor data 127 obtained from oneor more of the sensor devices 103) in a buffer.

A serial event routine may be performed that identifies which one of thesensor devices 103 transmitted sensor data 127 received by the clientdevice 121 (e.g., using a unique sensor identifier). The sensor data 127may be stored in association with the sensor device 103 thatcommunicated the sensor data 127 as well as a time the sensor data 127was obtained. A call to a quaternion processing routine may be called toprocess quaternions received from a particular sensor device 103.

The quaternion processing routine may be called to create a string ofcharacters of the sensor data 127 received from the particular sensordevice 103. As the sensor data 127 is received from the sensor device103 as a stream of characters, the characters may be stored in a stringdata object and parsed using a “split( )” function. The quaternionsobtained from the particular sensor device 103 may be casted into floatsto allow the client application 158 to perform subsequent processingwhich might not be available were the quaternions stored as strings ofcharacters. An axis is identified from the quaternions as well as anangle of rotation. Using the axis and the angle of rotation, a functioncall may be performed to a “Draw( ) routine” that generates and/orupdates a three-dimensional reconstruction 150.

The “Draw( ) routine” may include a loop to constantly update thethree-dimensional reconstruction 150 until a terminate condition isidentifier. The routine may push orientation matrices that updateapplicable portions of the body 100 as well as rotate the orientationmatrices by the determined angle and axis of rotation.

Moving on to FIG. 12, shown is a flowchart that provides one example ofthe operation of a portion of the computing environment 124 according tovarious embodiments. It is understood that the flowchart of FIG. 12provides merely an example of the many different types of functionalarrangements that may be employed to implement the operation of theportion of the client device 121 as described herein. As an alternative,the flowchart of FIG. 12 may be viewed as depicting an example ofelements of a method implemented by the data processing application 139executable by a hardware processor of the computing environment 124according to one or more embodiments.

In various embodiments, the functions performed by the clientapplication 158 can be offloaded for processing by the computingenvironment 124. Beginning with 1203, the computing environment 124accesses a request received from the client device 121 for athree-dimensional reconstruction 150. In some embodiments, the requestmay include sensor data 127 received by the client device 121 from thesensor devices 103. Accordingly, in 1206, the computing environment 124accesses sensor data 127 received by the computing environment 124 from,for example, the client device 121 and the sensor devices 103.

As noted above, the sensor devices 103 may include a communicationmodule 118, such as a Bluetooth® module, a Bluetooth® low-energy module,a ZigBee® module, an NFC module, a Wi-Fi module, or other suitablecommunication module 118 capable of communicating with another computingdevice. In various embodiments, the sensor devices 103, or a mastersensor device 103, may be configured to communicate the sensor data 127to the computing environment 124 asynchronously. The data processingapplication 139 may be configured to access the sensor data 127 measuredby the sensor devices 103. The sensor data 127 may include the positionof the portion of the body 100 during the movement performed, or otherdata collected during the movement.

In 1209, the sensor data 127 collected from the client device 121 or thesensor devices 103 may be stored in a buffer or in the data store 136.To this end, in some embodiments, the sensor data 127 may be stored in adatabase (e.g., SQL database or SQL-Lite database) of the data store 136in association with a time the movement was performed.

Next, in 1212, the sensor data 127 may be processed to determine aposition and/or orientation of each of the sensors devices 103 based onthe sensor data 127 received from each of the sensor devices 103. Forexample, sensor data 127 received from a sensor device 103 at a shouldermay be processed to identify a position or orientation of the sensordevice 103 at the shoulder, and so forth. In some embodiments, thesensor data 127 is processed to monitor changes in a position of acorresponding sensor device 103, such as changes in left movement, rightmovement, upwards movements, downwards movement, forward movement,backwards movement, pitch, roll, and yaw.

Using the processed sensor data 127, in 1218, the three-dimensionalreconstruction 150 is generated or updated and is transmitted to theclient device 121 over the network 133 for rendering in the display 155.In one embodiment, to update the three-dimensional reconstruction 150, aprogrammatic function call or API call is performed to provide agraphics library with an updated position and/or orientation of thecorresponding portion of the body 100.

Further, in some embodiments, the computing environment 124 (e.g., viathe data processing application 139) may be configured to determinewhether a movement performed by a body 100, having the sensor devices103 coupled thereon, conforms to an ideal movement, such as a movementassigned by a medical professional, a movement performed by a coach orathlete, or other predefined movement. Accordingly, in 1221, themovement performed by the body 100 having the sensor devices 103 coupledthereon is compared to ideal movement data 142 representative of anideal movement.

The ideal movement data 142 may reflect sensor data 127 collected by aperson who performed an ideal movement. As a result, a comparison ofsensor data 127 to sensor data 127 is performed. In other words, sensordata 127 collected when a first person performed a movement is comparedto sensor data 127 collected when a second person performed the samemovement. A position of a sensor device 103 on the first person may becompared to a position of a sensor device 103 on a second person duringat a same point in time during the movement.

In 1224, a determination is made whether a first movement (e.g., amovement performed by a wearer of the sensor devices 103) conforms to asecond movement (e.g., ideal movement data 142). If a first movement iswithin a predefined threshold of a second movement, no action may betaken and the process may proceed to 1227. However, in some embodiments,a notification may be generated for rendering on a display 155 thatindicates that the movement performed by a user conforms with idealmovement data 142 (e.g., “Good job, your movement is correct!”).

Referring back to 1224, if the first movement does not conform to thesecond movement (e.g., the first position is not within a predefinedthreshold of the second movement), the process proceeds to 1230 where asuggested change 145 in the movement may be generated for rendering inthe display 155. The suggested change 145 may include, for example, achange in the position of the portion of the body 100 where a particularsensor device 103 is attached that, if performed as suggested, wouldconform to the ideal movement data 142. As may be appreciated, the userof the client device 121 may then attempt to perform a subsequentmovement based on the suggested change 145 to conform to an idealmovement.

The process then proceeds to 1227 where a determination is made whethera terminate condition has been identified. In some embodiments, aterminate condition may occur after a predefined amount of time, after acomplete movement has been performed (e.g., a complete pitch motion, acomplete jump shot, or a complete golf swing), when a user closes theclient application 158, or when a user pauses the collection of sensordata 127 in the client application 158. If a terminate condition is notidentified, the process reverts back to 1009 to continue accessingcollected sensor data 127, for example, to continue updating thethree-dimensional reconstruction 150 and to continue generatingsuggested changes 145 in movement. If a terminate condition isidentified, however, the process may proceed to completion.

With reference to FIG. 13, shown is a schematic block diagram of asensor device 103 according to an embodiment of the present disclosure.The computing environment 124 includes one or more microcontroller 109.Each microcontroller 109 includes at least one processor circuit, forexample, having a processor 1303 and a memory 1306, both of which arecoupled to a local interface 1309. The local interface 1309 maycomprise, for example, a data bus with an accompanying address/controlbus or other bus structure as can be appreciated. The sensor device 103may further comprise one or more accelerometers 112, gyroscopes 115,magnetometers 117, and/or other components.

Stored in the memory 1306 are both data and several components that areexecutable by the processor 1303. In particular, stored in the memory1306 and executable by the processor 1303 is a microcontrollerapplication 1312, and potentially other applications. Also stored in thememory 1306 may be a sensor data store 1315 and other data. In addition,an operating system may be stored in the memory 1306 and executable bythe processor 1303.

Turning now to FIG. 14, shown is a schematic block diagram of the clientdevice 121 according to an embodiment of the present disclosure. Theclient device 121 includes one or more computing devices. Each computingdevice includes at least one processor circuit, for example, having aprocessor 1403 and a memory 1406, both of which are coupled to a localinterface 1409. To this end, each computing device may comprise, forexample, a mobile computing device, such as a smartphone, a tablet, asmartwatch, or other similar device. The local interface 1409 maycomprise, for example, a data bus with an accompanying address/controlbus or other bus structure as can be appreciated.

Stored in the memory 1406 are both data and several components that areexecutable by the processor 1403. In particular, stored in the memory1406 and executable by the processor 1403 are the client application158, and potentially other applications. Also stored in the memory 1406may be a client device data store 1412 and other data. In addition, anoperating system may be stored in the memory 1406 and executable by theprocessor 1403.

With reference to FIG. 15, shown is a schematic block diagram of thecomputing environment 124 according to an embodiment of the presentdisclosure. The computing environment 124 includes one or more computingdevices 1500. Each computing device 1500 includes at least one processorcircuit, for example, having a processor 1503 and a memory 1506, both ofwhich are coupled to a local interface 1509. To this end, each computingdevice 1500 may comprise, for example, at least one server computer orlike device. The local interface 1509 may comprise, for example, a databus with an accompanying address/control bus or other bus structure ascan be appreciated. It is understood that there may be otherapplications that are stored in the memory 1506 and are executable bythe processor 1503 as can be appreciated.

Where any component discussed herein is implemented in the form ofsoftware, any one of a number of programming languages may be employedsuch as, for example, C, C++, C#, Objective C, Java®, JavaScript®, Perl,PHP, Visual Basic®, Python®, Ruby, Flash®, or other programminglanguages.

A number of software components are stored in the memory 1506 and areexecutable by the processor 1503. In this respect, the term “executable”means a program file that is in a form that can ultimately be run by theprocessor 1503. Examples of executable programs may be, for example, acompiled program that can be translated into machine code in a formatthat can be loaded into a random access portion of the memory 1506 andrun by the processor 1503, source code that may be expressed in properformat such as object code that is capable of being loaded into a randomaccess portion of the memory 1506 and executed by the processor 1503, orsource code that may be interpreted by another executable program togenerate instructions in a random access portion of the memory 1506 tobe executed by the processor 1503, etc. An executable program may bestored in any portion or component of the memory 1506 including, forexample, random access memory (RAM), read-only memory (ROM), hard drive,solid-state drive, USB flash drive, memory card, optical disc such ascompact disc (CD) or digital versatile disc (DVD), floppy disk, magnetictape, or other memory components.

The memory 1506 is defined herein as including both volatile andnonvolatile memory and data storage components. Volatile components arethose that do not retain data values upon loss of power. Nonvolatilecomponents are those that retain data upon a loss of power. Thus, thememory 1506 may comprise, for example, random access memory (RAM),read-only memory (ROM), hard disk drives, solid-state drives, USB flashdrives, memory cards accessed via a memory card reader, floppy disksaccessed via an associated floppy disk drive, optical discs accessed viaan optical disc drive, magnetic tapes accessed via an appropriate tapedrive, and/or other memory components, or a combination of any two ormore of these memory components. In addition, the RAM may comprise, forexample, static random access memory (SRAM), dynamic random accessmemory (DRAM), or magnetic random access memory (MRAM) and other suchdevices. The ROM may comprise, for example, a programmable read-onlymemory (PROM), an erasable programmable read-only memory (EPROM), anelectrically erasable programmable read-only memory (EEPROM), or otherlike memory device.

Also, the processor 1503 may represent multiple processors 1503 and/ormultiple processor cores and the memory 1506 may represent multiplememories 1506 that operate in parallel processing circuits,respectively. In such a case, the local interface 1509 may be anappropriate network that facilitates communication between any two ofthe multiple processors 1503, between any processor 1503 and any of thememories 1506, or between any two of the memories 1506, etc. The localinterface 1509 may comprise additional systems designed to coordinatethis communication, including, for example, performing load balancing.The processor 1503 may be of electrical or of some other availableconstruction.

Although the microcontroller application 1312, the client application158, the data processing application 139, and other various systemsdescribed herein may be embodied in software or code executed by generalpurpose hardware as discussed above, as an alternative the same may alsobe embodied in dedicated hardware or a combination of software/generalpurpose hardware and dedicated hardware. If embodied in dedicatedhardware, each can be implemented as a circuit or state machine thatemploys any one of or a combination of a number of technologies. Thesetechnologies may include, but are not limited to, discrete logiccircuits having logic gates for implementing various logic functionsupon an application of one or more data signals, application specificintegrated circuits (ASICs) having appropriate logic gates,field-programmable gate arrays (FPGAs), or other components, etc. Suchtechnologies are generally well known by those skilled in the art and,consequently, are not described in detail herein.

The flowcharts of FIGS. 8, 10, and 12 show the functionality andoperation of an implementation of portions of the microcontrollerapplication 1312, the client application 158, and the data processingapplication 139. If embodied in software, each block may represent amodule, segment, or portion of code that comprises program instructionsto implement the specified logical function(s). The program instructionsmay be embodied in the form of source code that comprises human-readablestatements written in a programming language or machine code thatcomprises numerical instructions recognizable by a suitable executionsystem such as a processor 1503 in a computer system or other system.The machine code may be converted from the source code, etc. If embodiedin hardware, each block may represent a circuit or a number ofinterconnected circuits to implement the specified logical function(s).

Although the flowcharts of FIGS. 8, 10, and 12 show a specific order ofexecution, it is understood that the order of execution may differ fromthat which is depicted. For example, the order of execution of two ormore blocks may be scrambled relative to the order shown. Also, two ormore blocks shown in succession in FIGS. 8, 10, and 12 may be executedconcurrently or with partial concurrence. Further, in some embodiments,one or more of the blocks shown in FIGS. 8, 10, and 12 may be skipped oromitted. In addition, any number of counters, state variables, warningsemaphores, or messages might be added to the logical flow describedherein, for purposes of enhanced utility, accounting, performancemeasurement, or providing troubleshooting aids, etc. It is understoodthat all such variations are within the scope of the present disclosure.

Also, any logic or application described herein, including themicrocontroller application 1312, the client application 158, the dataprocessing application 139, that comprises software or code can beembodied in any non-transitory computer-readable medium for use by or inconnection with an instruction execution system such as, for example, aprocessor 1503 in a computer system or other system. In this sense, thelogic may comprise, for example, statements including instructions anddeclarations that can be fetched from the computer-readable medium andexecuted by the instruction execution system. In the context of thepresent disclosure, a “computer-readable medium” can be any medium thatcan contain, store, or maintain the logic or application describedherein for use by or in connection with the instruction executionsystem.

The computer-readable medium can comprise any one of many physical mediasuch as, for example, magnetic, optical, or semiconductor media. Morespecific examples of a suitable computer-readable medium would include,but are not limited to, magnetic tapes, magnetic floppy diskettes,magnetic hard drives, memory cards, solid-state drives, USB flashdrives, or optical discs. Also, the computer-readable medium may be arandom access memory (RAM) including, for example, static random accessmemory (SRAM) and dynamic random access memory (DRAM), or magneticrandom access memory (MRAM). In addition, the computer-readable mediummay be a read-only memory (ROM), a programmable read-only memory (PROM),an erasable programmable read-only memory (EPROM), an electricallyerasable programmable read-only memory (EEPROM), or other type of memorydevice.

Further, any logic or application described herein, including themicrocontroller application 1312, the client application 158, the dataprocessing application 139, may be implemented and structured in avariety of ways. For example, one or more applications described may beimplemented as modules or components of a single application. Further,one or more applications described herein may be executed in shared orseparate computing devices or a combination thereof. For example, aplurality of the applications described herein may execute in the samecomputing device, or in multiple computing devices in the same computingenvironment 124. Additionally, it is understood that terms such as“application,” “service,” “system,” “engine,” “module,” and so on may beinterchangeable and are not intended to be limiting.

Disjunctive language such as the phrase “at least one of X, Y, or Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y, or Z, or any combination thereof (e.g., X, Y, and/or Z).Thus, such disjunctive language is not generally intended to, and shouldnot, imply that certain embodiments require at least one of X, at leastone of Y, or at least one of Z to each be present.

It should be emphasized that the above-described embodiments of thepresent disclosure are merely possible examples of implementations setforth for a clear understanding of the principles of the disclosure.Many variations and modifications may be made to the above-describedembodiment(s) without departing substantially from the spirit andprinciples of the disclosure. All such modifications and variations areintended to be included herein within the scope of this disclosure andprotected by the following claims.

Therefore, the following is claimed:
 1. A system for telemedicinebetween a patient and a remotely-located medical professional,comprising: a plurality of sensor devices in data communication with aclient device, the client device comprising at least one hardwareprocessor, wherein individual ones of the sensor devices comprise: aband to secure a respective one of the sensor devices to a portion of abody of a patient, the respective one of the sensor devices beingsecured to the portion of the body of the patient based at least in parton a predefined arrangement of sensor devices stored in a memory of theclient device; circuitry comprising an accelerometer and a gyroscopethat measures a position of the portion of the body of the patient towhich the band is secured during a movement performed by the portion ofthe body of the patient; and the client device; and an applicationexecuted in the client device, the application comprising programinstructions that, when executed, direct the client device to: establisha telemedicine session between the client device and a remote computingdevice operated by a medical professional located remotely from thepatient; render a first three-dimensional graphical model of the body ofthe patient for viewing in a display accessible to the client device;receive from the remote computing device an indication of a movement tobe performed, the movement to be performed being assigned by the medicalprofessional; access sensor data measured by the plurality of sensordevices, the sensor data comprising the position of the portion of thebody of the patient during the movement; update the firstthree-dimensional graphical model of the body of the patient in thedisplay using the sensor data to further comprise the movement; receive,from the remote computing device, ideal sensor data comprising aposition of a portion of a body of the medical professional during anideal body movement performed by the portion of the body of the medicalprofessional, the ideal sensor data being generated by at least onesensor device secured to the portion of the body of the medicalprofessional during the ideal body movement and stored in a data storeof the remote computing device; render a second three-dimensionalgraphical model of the body of the medical professional in the displaybased at least in part on the ideal sensor data, the secondthree-dimensional graphical model comprising the ideal body movement;determine whether the movement conforms to the ideal body movement basedat least in part on a difference between a position of at least onesensor device of the plurality of sensor devices from the sensor dataand a position of the at least one sensor device secured to the portionof the body of the medical professional from the ideal sensor data at agiven time exceeding a predefined threshold; in response to the movementnot conforming to the ideal body movement, identify a suggested changein the movement that conforms to the ideal body movement; send acommunication to the remote computing device, the communicationcomprising an indication that the movement does not conform to the idealbody movement; and render an indication of the suggested change in themovement in the display.
 2. The system of claim 1, wherein the circuitrythat measures the position of the portion of the body of the patient towhich the band is secured further comprises a magnetometer.
 3. Thesystem of claim 2, wherein at least one of the sensor devices comprisesa master sensor device and other ones of the sensor devices comprise aplurality of slave sensor devices, the master sensor device beingconfigured to obtain the sensor data from the slave sensor devices andcommunicate the sensor data to the client device via a communicationmodule.
 4. The system of claim 1, wherein the program instructions, whenexecuted, further direct the client device to communicate the sensordata to a server computing device over a network for remote processing.5. The system of claim 1, wherein the sensor devices are configured tocommunicate the sensor data to the client device asynchronously.
 6. Asystem for telemedicine between a patient and a remotely-located medicalprofessional comprising: a plurality of sensor devices in datacommunication with a client device, the client device comprising atleast one hardware processor, wherein individual ones of the sensordevices comprise: a band to secure a respective one of the sensordevices to a portion of a body of a patient, the respective one of thesensor devices being secured to the portion of the body of the patientbased at least in part on a predefined arrangement of sensor devicesstored in a memory of the client device; circuitry comprising anaccelerometer and a gyroscope that measures a position of the portion ofthe body of the patient to which the band is secured during a movementperformed by the portion of the body of the patient; and an applicationexecuted in a remote computing device in data communication with theclient device over a network, the application comprising logic that:establishes a telemedicine session between the client device and aremote computing device operated by a medical professional locatedremotely from the patient; generates a first three-dimensional graphicalmodel of the body of the patient; accesses sensor data measured by theplurality of sensor devices received from the client device over thenetwork, the sensor data comprising the position of the portion of thebody of the patient during the movement; updates the firstthree-dimensional graphical model of the body of the patient using thesensor data to further comprise the movement; receives, from the remotecomputing device operated by the medical professional, ideal sensor datacomprising a position of a portion of a body of the medical professionalduring an ideal body movement, the ideal sensor data being generated byat least one sensor device secured to the portion of the body of themedical professional and stored in a data store of the remote computingdevice; generates a second three-dimensional graphical model of the bodyof the medical professional based at least in part on the ideal sensordata, the second three-dimensional graphical model comprising the idealbody movement; determines whether the movement conforms to the idealbody movement based at least in part on a difference between a positionof at least one sensor device of the plurality of sensor devices fromthe sensor data and a position of the at least one sensor device securedto the portion of the body of the medical professional from the idealsensor data at a given time exceeding a predefined threshold; inresponse to the movement not conforming to the ideal body movement,identifies a suggested change in the movement that conforms to the idealbody movement; sends a communication to the remote computing device, thecommunication comprising an indication that the movement does notconform to the ideal body movement; and causes transmission of the firstthree-dimensional graphical model of the body of the patient, the secondthree-dimensional graphical model, and the suggested change in themovement to the remote computing device for rendering in a display. 7.The system of claim 6, wherein the circuitry that measures the positionof the portion of the body of the patient to which the band is securedfurther comprises a magnetometer.
 8. The system of claim 6, wherein atleast one of the sensor devices comprises a master sensor device andother ones of the sensor devices comprising a plurality of slave sensordevices, the master sensor device being configured to obtain the sensordata from the slave sensor devices and communicate the sensor data tothe client device via a communication module.
 9. The system of claim 6,wherein the application further comprises logic that communicates thesensor data to a server computing device over a network for remoteprocessing.
 10. The system of claim 6, wherein the sensor devices areconfigured to communicate the sensor data to the client deviceasynchronously.
 11. A computer-implemented method for telemedicinebetween a patient and a remotely-located medical professional,comprising: establishing, by at least one computing device comprising atleast one hardware processor, a telemedicine session between the atleast one computing device and a remote computing device operated by amedical professional located remotely from a patient operating the atleast one computing device; generating, by the at least one computingdevice, a first three-dimensional graphical model of a body of thepatient; receiving, by the at least one computing device, an indicationof a movement to be performed from the remote computing device, themovement to be performed being assigned by the medical professional;accessing, by the at least one computing device, sensor data obtained bya plurality of sensor devices, wherein individual ones of the sensordevices comprise: a band to secure a respective one of the sensordevices to a portion of a body of the patient, the respective one of thesensor devices being secured to the portion of the body of the patientbased at least in part on a predefined arrangement of sensor devicesstored in a memory of the at least one computing device; and circuitrycomprising an accelerometer and a gyroscope that measures a position ofthe portion of the body of the patient to which the band is securedduring a movement performed by the portion of the body of the patient;updating, by the at least one computing device, the firstthree-dimensional graphical model of the body of the patient using thesensor data to further comprise the movement performed; receiving, bythe at least one computing device, predefined sensor data from theremote computing device, ideal sensor data comprising a position of aportion of a body of the medical professional during an ideal bodymovement performed by the portion of the body of the medicalprofessional, the predefined sensor data being generated by at least onesensor device secured to the portion of the body of the medicalprofessional and stored in a data store of the remote computing device;generating, by the at least one computing device, a secondthree-dimensional graphical model of the body of the medicalprofessional based at least in part on the ideal sensor data, the secondthree-dimensional graphical model comprising the ideal body movement;determining whether the movement conforms to the ideal body movementbased at least in part on a difference between a position of at leastone sensor device of the plurality of sensor devices from the sensordata and a position of the at least one sensor device secured to theportion of the body of the medical professional from the predefinedsensor data at a given time exceeding a predefined threshold; inresponse to the movement not conforming to the ideal body movement,identifying a suggested change in the movement that conforms to theideal body movement; sending, by the at least one computing device, acommunication to the remote computing device, the communicationcomprising an indication that the movement does not conform to the idealbody movement; and causing the first three-dimensional graphical modelof the body, the second three-dimensional graphical model, and thesuggested change in the movement to be rendered in a display.
 12. Themethod of claim 11, wherein the circuitry that measures the position ofthe portion of the body of the patient to which the band is securedfurther comprises a magnetometer.
 13. The method of claim 12, wherein atleast one of the sensor devices comprises a master sensor device andother ones of the sensor devices comprising a plurality of slave sensordevices, the master sensor device being configured to obtain the sensordata from the slave sensor devices and communicate the sensor data to aclient device via a communication module.
 14. The method of claim 13,wherein the communication module further comprises a BLUETOOTH® module,a BLUETOOTH® low-energy module, a ZIGBEE® module, or a near fieldcommunication (NFC) module.
 15. The method of claim 11, furthercomprising causing, by the at least one computing device, transmissionof the sensor data to a server computing device over a network forremote processing.
 16. The method of claim 11, wherein the predefinedarrangement further comprises: a first one of the sensor devices beinglocated at a first shoulder of the body; a second one of the sensordevices being located at a second shoulder of the body; a third one ofthe sensor devices being located at a first elbow of the body; a fourthone of the sensor devices being located at a second elbow of the body; afifth one of the sensor devices being located at a first wrist of thebody; a sixth one of the sensor devices being located at a second wristof the body; a seventh one of the sensor devices being located at afirst knee of the body; an eighth one of the sensor devices beinglocated at a second knee of the body; a ninth one of the sensor devicesbeing located at a first ankle of the body; a tenth one of the sensordevices being located at a second ankle of the body; an eleventh one ofthe sensor devices being located at a first side of hips of the body;and a twelfth one of the sensor devices being located at a second sideof hips of the body.
 17. The system of claim 6, wherein theaccelerometer is a first accelerometer, and the circuitry that measuresthe position of the portion of the body of the patient to which the bandis secured further comprises a second accelerometer and a thirdaccelerometer.
 18. The system of claim 6, wherein the gyroscope is afirst gyroscope, and the circuitry that measures the position of theportion of the body of the patient to which the band is secured furthercomprises a second gyroscope and a third gyroscope.
 19. The system ofclaim 1, wherein the first three-dimensional graphical model of the bodyof the patient and the second three-dimensional graphical model of thebody of the medical professional are rendered side-by-side in thedisplay.
 20. The system of claim 1, wherein the second three-dimensionalgraphical model of the body of the medical professional is rendered asan overlay on the first three-dimensional graphical model of the body ofthe patient in the display.